Recently, digital cameras as one of image input means to a personal computer have played an important role along with the spread of personal computers (to be simply referred to as “PCs” hereinafter) to the consumer market.
In a film camera, the white balance of a photographed image is adjusted by mounting a filter on a lens by the user (photographer) in accordance with the color temperature of a light source for irradiating an object to be photographed. In a digital camera, white balance processing (calculation processing) is executed to pickup image data output from an image pickup element having a color filter, which reduces the burden on the user.
White balance processing in the digital camera includes the following auto white balance processing and preset white balance processing.
(Auto White Balance Processing)
A white (achromatic) portion of pickup image data output from an image pickup element is detected. The color temperature of an irradiation light source with respect to the object in the pickup image is obtained from the detection result (white portion). The pickup image data output from the image pickup element is multiplied by a white balance control value (control value for changing the object region of the white portion into an achromatic color) based on the color temperature.
(Preset White Balance Processing)
Multiplication processing (white balance calculation processing) is performed to pickup image data output from the image pickup element so as to change image data of the object region into an achromatic color.
Preset white balance processing has a sunlight white balance mode in which a white balance control value corresponding to the color temperature of sunlight is selected when an object irradiated with sunlight as a light source is to be picked up, and a tungsten white balance mode in which a white balance control value corresponding to the color temperature of light from a tungsten electric light bulb is selected when an object irradiated by light from the tungsten electric light bulb as a light source is to be picked up.
FIGS. 4A and 4B show the arrangement of an image processing system 500 having the function of the above-described white balance processing.
As shown in FIGS. 4A and 4B, the image processing system 500 comprises a digital camera 600 and image reproducing apparatus 700.
The digital camera 600 has a function of storing image data from an image pickup device 603 to a memory 608.
The image reproducing apparatus 700 is a personal computer, development processing-dedicated apparatus, or the like. The image reproducing apparatus 700 has a function of reading out image data stored in the memory 608, performing development processing including white balance processing to the image data, and storing the processed image in a built-in memory 737.
A series of operations of the image processing system 500 will be explained.
In the digital camera 600, the image pickup device 603 outputs image data (data output from a color filter) 604 picked up by an image pickup element (not shown) having a color filter.
The pickup image data 604 output from the image pickup device 603 is stored in the memory 608 on the basis of the operation mode of the digital camera 600 as follows.
For example, when the operation mode of the digital camera 600 is a RAW mode, a lossless compressing/encoding processing device 605 losslessly compresses and encodes the pickup image data 604 which is output from the image pickup device 603 and does not undergo any processing. The losslessly compressed/encoded image data is directly stored in the memory 608.
When the operation mode of the digital camera 600 is a JPEG mode, a white balance control device 619 executes calculation processing for white balance control to the pickup image data 604 which is output from the image pickup device 603 and does not undergo any processing.
A development processing device 620 performs development processing such as color matrix transformation to the image data processed by the white balance control device 619. The development processing device 620 further performs lossy compression/encoding processing to generate JPEG image data.
An image output device 621 stores the JPEG image data obtained by the development processing device 620 in the memory 608.
The operation in the JPEG mode will be explained in detail. The white balance control device 619 executes calculation processing for the above-mentioned white balance control based on a white balance control value 618. The white balance control value 618 is generated as follows.
An auto white balance (AWB) calculating device 609 calculates an AWB control value 610 by performing white determination processing to each color component data (data output from a color filter) which constitutes the pickup image data 604 output from the image pickup device 603. The AWB calculating device 609 outputs the AWB control value 610 to a WB mode selecting device 616.
The WB mode selecting device 616 receives WB control values 612(1) to 612(N) together with the AWB control value 610 (N: integer of N≧2).
The WB control values 612(1) to 612(N) are white balance control values corresponding to the color temperatures of light sources which irradiate various objects to be picked up. The WB control values 612(1) to 612(N) are calculated by adjustment calculating devices 650(1) to 650(N).
More specifically, the adjustment calculating devices 650(1) to 650(N) calculate the WB control values 612(1) to 612(N) by executing calculation processing which absorbs the color filter characteristics of the image pickup elements of the following two digital cameras on the basis of reference WB control values 611(1) to 611(N) obtained from the results of picking up an object light source by a reference digital camera, reference image data 653 obtained by picking up a light source having an arbitrary fixed color temperature by the reference digital camera, and adjustment image data 654 obtained by picking up this light source by the digital camera 600. For example, the adjustment calculating device 650(x) calculates the WB control value 612(x) adjusted for the digital camera by executing calculation processing of setting a correction function derived using the reference WB control value 611(x) from the difference between the reference image data 653 and the adjustment image data 654 as a function of correcting the characteristics of the reference digital camera and digital camera 600.
The WB control values 612(1) to 612(N) calculated by the adjustment calculating devices 650(1) to 650(N) in this way are supplied to the WB mode selecting device 616 together with the AWB control value 610.
The reference image data 653 and adjustment image data 654 are stored in the memory 608 so as to allow the image reproducing apparatus 700 to use them.
The WB mode selecting device 616 outputs a control value 618 selected from the AWB control value 610 and the WB control values 612(1) to 612(N).
For example, when the digital camera 600 is set to the auto white balance mode, the WB mode selecting device 616 outputs the AWB control value 610 to the white balance control device 619.
The white balance control device 619 executes white balance control processing of the pickup image data 604 by multiplying, based on the control value 618 from the white balance control device 619, a gain and each color component data which constitutes the pickup image data 604 output from the image pickup device 603.
The development processing device 620 performs development processing (development processing of a luminance signal Y and color difference signal Pr/Pb) such as color matrix transformation to the image data processed by the white balance control device 619. Further, the development processing device 620 executes lossy compression/encoding processing to generate JPEG image data. The image output device 621 stores the JPEG image data in the memory 608.
The memory 608 is a removable memory which is inserted into a socket 607 of the digital camera 600. By inserting the memory 608 into a socket 723 of the image reproducing apparatus 700, data stored in the memory 608 is loaded into the image reproducing apparatus 700.
The image reproducing apparatus 700 reproduces the stored data (image data) loaded from the memory 608 as follows.
For example, to reproduce image data stored in the memory 608 in the RAW mode, a decompressing/decoding device 725 decodes image data 724 read out from the memory 608 to generate uncompressed original image data 740 of each color component output from the image pickup device 603 of the digital camera 600.
A white balance control device 726 performs calculation processing for white balance control on the basis of a white balance control value 734 to the image data 740 obtained by the decompressing/decoding device 725, similar to the white balance control device 619 of the digital camera 600.
More specifically, adjustment calculating devices 756(1) to 756(N) calculate WB control values 728(1) to 728(N) on the basis of reference WB control values 727(1) to 727(N), similar to the reference WB control values 611(1) to 611(N) in the digital camera 600, and reference image data and adjustment image data 759 stored in the memory 608 (the reference image data 653 and adjustment image data 654 used in the digital camera 600).
An auto white balance (AWB) calculating device 738 calculates an AWB control value 739 by performing white determination processing to each color component data (data output from a color filter) which constitutes the image data 740 obtained by the decompressing/decoding device 725.
A WB mode selecting device 733 outputs a control value 734 selected from the WB control values 728(1) to 728(N) calculated by the adjustment calculating devices 756(1) to 756(N) and the AWB control value 739 calculated by the AWB calculating device 738.
For example, when the image reproducing apparatus 700 is set to the auto white balance mode, the WB mode selecting device 733 outputs the AWB control value 739 as the control value 734.
The white balance control device 726 executes calculation processing for white balance control to the image data 740 from the decompressing/decoding device 725 on the basis of the control value 734 output from the WB mode selecting device 733.
A development processing device 635 performs development processing such as color matrix transformation to the image data processed by the white balance control device 726, similar to the development processing device 620 of the digital camera 600. In addition, the development processing device 635 executes lossy compression/encoding processing to generate JPEG image data. An image output device 736 stores the JPEG image data in the memory 737.
The conventional image processing system 500 shown in FIGS. 4A and 4B suffers the following problems.
For example, the image reproducing apparatus 700 must hold the same reference WB control values 727(1) to 727(N) as the reference WB control values 611(1) to 611(N) of the digital camera 600 in order to perform white balance control to image data stored in the memory 608 in the RAW mode by the digital camera 600, to perform development processing, to JPEG-encode the image data, and to store the JPEG image data. Calculation processing of calculating the control value 734 for white balance control by using the reference WB control values 727(1) to 727(N) and the reference image data 653 and adjustment image data 654 stored in the memory 608 by the digital camera 600 increases the calculation burden on the image reproducing apparatus 700, prolonging the calculation time.
Also, calculation processing of calculating the control value 739 for auto white balance control when the image reproducing apparatus 700 is set to the auto white balance mode increases the calculation burden on the image reproducing apparatus 700, prolonging the calculation time.
If, for example, the number of corresponding color temperatures and the number of reference WB control values are increased in white balance control in the digital camera 600, the number of reference WB control values must also be increased in the image reproducing apparatus 700 in order to cope with the change in the digital camera 600. This makes maintenance of the image reproducing apparatus 700 very difficult.